Continuous-time markov decision processes with nonzero terminal reward

In this article we consider a continuous-time Markov decision process with a denumerable state space and nonzero terminal rewards. We first establish the necessary and sufficient optimality condition without any restriction on the cost functions. The necessary condition is derived through the Pontryagin maximum principle and the sufficient condition, by the inherent structure of the problem. We introduce a dynamic programming approximation algorithm for the finite-horizon problem. As the time between discrete points decreases, the optimal policy of the discretized problem converges to that of the continuous-time problem in the sense of weak convergence. For the infinite-horizon problem, a successive approximation method is introduced as an alternative to a policy iteration method.     

An analysis of constrained robust regression estimators

Multicollinearity and nonnormal errors are problems often encountered in the application of linear regression. Estimators are proposed for dealing with the simultaneous occurrence of both multicollinearity and nonnormality. These estimators are developed by combining biased estimation techniques with certain robust criteria. An iteratively reweighted least-squares procedure is used to compute the estimates. The performance of the combined estimators is studied empirically through Monte Carlo experiments structured according to factorial designs. With respect to a mean-squared-error criterion, the combined estimators are superior to ordinary least-squares, pure biased estimators, and pure robust estimators when multicollinearity and nonnormality are present. The loss in efficiency for the combined estimators relative to least squares is small when these problems do not occur. Some guidelines for the use of these combined estimators are given.     

Multiechelon repairable-item provisioning in a time-varying environment using the randomization technique

Multiechelon repairable-item provisioning systems are considered under a time-varying environment. Such conditions could arise, for example, in a military context where a shift from peacetime operation to wartime operation takes place; or, in a civilian setting where a public transit system decides to increase its hours of operation or frequency of service. Exact Markovian models, incorporating a finite population of repairable components and limited repair capacity (nonample service), are treated, with transient solutions obtained using the randomization technique. The exact models are compared with the approximate Dyna-METRIC model which assumes an infinite population of components and ample repair capacity.     

Quadratic forms in spherical random variables: Generalized noncentral x2 distribution

Let X denote a random vector with a spherically symmetric distribution. The density of U = X'X, called a “generalized chi-square,” is derived for the noncentral case, when μ = E(X) ≠ 0. Explicit series representations are found in certain special cases including the “generalized spherical gamma,” the “generalized” Laplace and the Pearson type VII distributions. A simple geometrical representation of U is shown to be useful in generating random U variates. Expressions for moments and characteristic functions are also given. These densities occur in offset hitting probabilities.     

An inventory model with finite-range stochastic lead times

This article considers an inventory model with constant demand and stochastic lead times distributed over a finite range. A generalization of the EOQ formula with backorders is derived and ranges for the decision variables are obtained. The results are illustrated with the case of uniformly distributed lead time.     

A characterization of the wald distribution

Suppose X is a random variable having an absolutely continuous distribution function F(x). We assume that F(x) has the Wald distribution. A relation between the probability density function of X⁻¹ with that of X is used to characterize the Wald distribution.     

The capital supply curve in capacity expansion models: Some economic and algorithmic aspects

Capacity expansion models typically minimize the discounted cost of acquisition and operation over a given planning horizon. In this article we generalize this idea to one in which a capital supply curve replaces the usual discount rate. A capital supply curve is a means to model financial outlook, investment limits, and risk. We show that when such a curve is included in a capacity expansion model, it will, under certain conditions, provide a less capital intensive solution than one which incorporates a discount rate. In this article, we also provide an algorithm that solves capacity expansion models that incorporate a capital supply curve. The attractive feature of this algorithm is that it provides a means to utilize the “discount rate” models efficiently. Throughout, we give applications in power generation planning and computational experience for this application is also presented.